[0050] CLAIMS 

What is claimed is: 

1. A method comprising: 

matching outer and inner facial features of a face model with: 

that of a first model for a first face image; and 

that of a second model for a second face image; and 
adjusting each said matching outer and inner facial feature of the first and second models 
using the corresponding epipolar constraint for the first and second models. 

2. The method as defined in Claim 1, wherein: 

the inner facial features comprise a nose, a mouth and a pair of eyes; and 

the outer facial features comprise an arcuate bottom edge of the face and a pair of eyebrows. 

3. The method as defined in Claim 1, wherein a plurality points correspond to each said inner 
facial feature and comprise: 

a plurality of mouth corners corresponding to the mouth; 
a plurality of eyes comers corresponding to the eye; and 
a nose tip corresponding to the nose. 
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4. The method as defined in Claim 1, wherein the first and second face images are the same face 
captured at the substantially the same respective camera orientation. 

5. The method as defined in Claim 4, wherein the respective face poses of the first and second 
face images are: 

substantially the same as to pitch; 
substantially the same as to roll; and 

different as to yaw in a range from about 3 degrees to about 10 degrees. 

6. The method as defined in Claim 1, wherein the first and second face images are pictures taken 
at the substantially the same respective camera orientation of the same face in substantially the same 
face pose range but differing by the yaw. 

7. The method as defined in Claim 1, wherein the face pose range differs in the first and second 
face images by a yaw different of not more than about ten degrees (10°). 

8. The method as defined in Claim 1, wherein: 

the first face image is a full frontal picture of a human face; and 

the second face image differs from the first face image by a yaw of not more than about ten 
degrees (10°). 

9. The method as defined in Claim 1, further comprising outputting a representation of each of the 
first and second models, each including the respective inner and outer facial features thereof 
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10. The method as defined in Claim 9, wherein the outputting further comprises: 
repeating the matching and the adjusting until each said match of the outer and inner facial 

features on: 

the first model is not greater than a predetermined convergence toleration with respect 
to that of the face model; and 

the second model is not greater than a predetermined convergence toleration with 
respect to that of the face model. 

11. The method as defined in Claim 1, further comprising repeating the matching and the 
adjusting until each said match of the outer and inner facial features on: 

the first model is not greater than a predetermined convergence toleration with respect to that 
of the face model; and 

the second model is not greater than a predetermined convergence toleration with respect to 
that of the face model. 

12. The method as defined in Claim 1, further comprising identifying the face of a person in a 
database that matches at least one of the first and second face images by using the respective outer 
and inner facial features of the first and second models. 

13. A computer readable medium comprising instructions that, when executed, perform the 
method of Claim 1. 
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14. A method comprising: 

matching outer and inner facial features of a face model, each being represented by plurality 
of points, with: 

that of a first model for a first face image; and 

that of a second model for a second face image; 
adjusting the plurality of points for each said matching outer and inner facial feature of the 
first and second models using the corresponding epipolar constraint for the first and second models; 
and 

repeating the matching and the adjusting until: 

a majority of the points for each of the outer and inner facial features on the first model 
that matches that of the face model has a relative offset there between of not greater than a 
predetermined convergence toleration; and 

a majority of the points for each of the outer and inner facial features on the first model 
that matches that of the face model has a relative offset there between not greater than the 
predetermined convergence toleration. 

15. The method as defined in Claim 14, further comprising outputting a representation of each of 
the first and second models, each including the respective inner and outer facial features thereof 

16. The method as defined in Claim 14, wherein: 

the inner facial features comprise a nose, a mouth and a pair of eyes; and 

the outer facial features comprise an arcuate bottom edge of the face and a pair of eyebrows. 
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17. The method as defined in Claim 14, wherein the plurality points and corresponding said inner 
facial feature are selected from the group consisting of: 

a plurality of mouth corners corresponding to a mouth; 

a plurality of eyes corners corresponding to an eye; and 

a nose tip corresponding to a nose. 



iee@hayes 



MSH523U$.P«.App..(toc 



18. The method as defined in Claim 14, wherein the first and second face images are the same 
face captured at the substantially the same respective camera orientation. 

19. The method as defined in Claim 18, wherein the respective face poses of the first and second 
face images are: 

substantially the same as to pitch; 
substantially the same as to roll; and 

different as to yaw in a range from about 3 degrees to about 10 degrees. 

20. The method as defined in Claim 14, wherein the first and second face images are pictures 
taken at the substantially the same respective camera orientation of the same face in substantially the 
same face pose range but differing by the yaw. 

21. The method as defined in Claim 14, wherein the face pose range differs in the first and 
second face images by a yaw different of not more than about ten degrees (10°). 

22. The method as defined in Claim 14, wherein: 

the first face image is a full frontal picture of a human face; and 

the second face image differs from the first face image by a yaw of not more than about ten 
degrees (10°). 
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23. The method as defined in Claim 14, further comprising identifying the face of a person in a 
database that matches at least one of the first and second face images by using the points that 
respectively correspond to the outer and inner facial features of the first and second models. 

24. A computer readable medium comprising instructions that, when executed, perform the 
method of Claim 14. 

25. A method comprising the steps of: 

(a) initializing, using a face model having outer and inner facial features each including a 
plurality points, first and second models for respective first and second face images, wherein each of 
the first and second models has outer and inner facial features each including a plurality points; 

(b) estimating a fundamental matrix for the first and second models; 

(c) updating the plurality of points for each said inner facial feature of the first and second 
models by using the respective local texture for each said point; 

(d) updating each said inner facial feature of the first and second models; 

(e) updating, using the fundamental matrix and the corresponding epipolar constraint, the 
plurality of points for each of the inner facial features of the first and second models; and 

(f) updating each said outer facial feature of the first and second models. 

26. The method as defined in Claim 25, wherein the steps further comprise: 

(g) if the first model has converged within a predetermined tolerance of the face model, 
outputting a representation of the inner and outer facial features of the first model, otherwise 
repeating steps (c) through (f) for the first model; and 
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(h) if the second model has converged within a predetermined tolerance of the face model, 
outputting a representation of the inner and outer facial features of the second model, otherwise 
repeating steps (c) through (f) for the second model. 

27. The method as defined in Claim 25, wherein: 

the inner facial features comprise a nose, a mouth and a pair of eyes; and 

the outer facial features comprise an arcuate bottom edge of the face and a pair of eyebrows. 

28. The method as defined in Claim 27, wherein the plurality points corresponding to the inner 
facial feature of: 

the mouth comprises a plurality of mouth corners; 
each said eye comprises a plurality of eyes corners; and 
the nose comprises a nose tip. 

29. The method as defined in Claim 25, wherein the first and second face images are the same 
face captured at the substantially the same respective camera orientation. 

30. The method as defined in Claim 29, wherein the respective face poses of the first and second 
face images are: 

substantially the same as to pitch; 
substantially the same as to roll; and 

different as to yaw in a range from about 3 degrees to about 10 degrees. 
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31. The method as defined in Claim 25, wherein the first and second face images are pictures 

taken at the substantially the same respective camera orientation of the same face in substantially the 

i 

same face pose range but differing by the yaw. 

32. The method as defined in Claim 25, wherein the face pose range differs in the first and 
second face images by a yaw different of not more than about ten degrees (10°). 

33. The method as defined in Claim 25, wherein: 

the first face image is a full frontal picture of a human face; and 

the second face image differs from the first face image by a yaw of not more than about ten 
degrees (10°). 

34. The method as defined in Claim 25, wherein the local texture for each said point comprises 
photometric data at or proximal to the point. 

35. The method as defined in Claim 25, further comprising identifying the face of a person in a 
database that matches at least one of the first and second face images by using, respectively, the 
output representation of the inner and outer facial features of: 

the first model; and 
the second model. 

36. A computer readable medium comprising instructions that, when executed, perform the 
method of Claim 25. 
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37. An apparatus comprising: 
memory including: 

one or more programs; 

outer and inner facial features of a face model, each being represented by a plurality of 

points; 

a first face image; and 
a second face image; 

one or more processors configured to execute the one or more programs in the memory so as 
to perform steps including: 

match the plurality of points of the outer and inner facial features of the face model 

with: 

that of a first model for the first face image; and 

that of a second model for the second face image; 
adjust the plurality of points for each said matching outer and inner facial feature of 
the first and second models using the corresponding epipolar constraint for the first and 
second models; and 

repeat the match and adjust steps until: 

a majority of the points for each of the outer and inner facial features on the 
first model that matches that of the face model has a relative offset there between of 
not greater than a predetermined convergence toleration; and 

a majority of the points for each of the outer and inner facial features on the 
first model that matches that of the face model has a relative offset there between not 
greater than the predetermined convergence toleration. 
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38. The apparatus as defined in Claim 37, wherein the steps further comprise outputting a 
representation of each of the first and second models, each including the respective inner and outer 
facial features thereof 

39. The apparatus as defined in Claim 37, wherein: 

the inner facial features comprise a nose, a mouth and a pair of eyes; and 

the outer facial features comprise an arcuate bottom edge of the face and a pair of eyebrows. 

40. The apparatus as defined in Claim 37, wherein the plurality points and corresponding said 
inner facial feature are selected from the group consisting of: 

a plurality of mouth corners corresponding to a mouth; 
a plurality of eyes corners corresponding to an eye; and 
a nose tip corresponding to a nose. 

41. The apparatus as defined in Claim 37, wherein the first and second face images are the same 
face captured at the substantially the same respective camera orientation. 

42. The apparatus as defined in Claim 41, wherein the respective face poses of the first and 
second face images are: 

substantially the same as to pitch; 
substantially the same as to roll; and 

different as to yaw in a range from about 3 degrees to about 10 degrees. 
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43. The apparatus as defined in Claim 37, wherein the first and second face images are pictures 
taken at the substantially the same respective camera orientation of the same face in substantially the 
same face pose range but differing by the yaw. 

44. The apparatus as defined in Claim 37, wherein the face pose range differs in the first and 
second face images by a yaw different of not more than about ten degrees (10°). 

45. The apparatus as defined in Claim 37, wherein: 

the first face image is a full frontal picture of a human face; and 

the second face image differs from the first face image by a yaw of not more than about ten 
degrees (10°). 

46. An apparatus comprising: 

means for initializing, using a face model having outer and inner facial features each including 
a plurality points, first and second models for respective first and second face images, wherein each 
of the first and second models has outer and inner facial features each including a plurality points; 

means for estimating a fundamental matrix for the first and second models; 

means for updating the plurality of points for each said inner and outer facial feature of the 
first and second models by using the respective local texture for each said point; 

means for updating each said inner facial feature of the first and second models; 

means for updating, using the fundamental matrix and the corresponding epipolar constraint, 
the plurality of points for each of the inner facial features of the first and second models; and 

means for updating, using the plurality of points for each of the inner facial features of the 
first and second models, each said outer facial feature of the first and second models. 




47. The apparatus as defined in Claim 46, further comprising: 

means, if a majority of the points for each of the outer and inner facial features on the first 
model that matches that of the face model has a relative offset there between of not greater than a 
predetermined convergence toleration, for outputting a representation of the inner and outer facial 
features of the first model; and 

means, if a majority of the points for each of the outer and inner facial features on the first 
model that matches that of the face model has a relative offset there between not greater than the 
predetermined convergence toleration, for outputting a representation of the inner and outer facial 
features of the second model. 

48. The apparatus as defined in Claim 46, wherein: 

the inner facial features comprise a nose, a mouth and a pair of eyes; and 

the outer facial features comprise an arcuate bottom edge of the face and a pair of eyebrows. 

49. The apparatus as defined in Claim 48, wherein the plurality points corresponding to the inner 
facial feature of: 

the mouth comprises a plurality of mouth corners; 
each said eye comprises a plurality of eyes corners; and 
the nose comprises a nose tip. 
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50. The apparatus as defined in Claim 46, wherein the first and second face images are the same 
face captured at the substantially the same respective camera orientation. 

51. The apparatus as defined in Claim 46, wherein the respective face poses of the first and 
second face images are: 

substantially the same as to pitch; 
substantially the same as to roll; and 

different as to yaw in a range from about 3 degrees to about 10 degrees. 

52. The apparatus as defined in Claim 46, wherein the first and second face images are pictures 
taken at the substantially the same respective camera orientation of the same face in substantially the 
same face pose range but differing by the yaw. 

53. The apparatus as defined in Claim 46, wherein the face pose range differs in the first and 
second face images by a yaw different of not more than about ten degrees (10°). 

54. The apparatus as defined in Claim 46, wherein: 

the first face image is a full frontal picture of a human face; and 

the second face image differs from the first face image by a yaw of not more than about ten 
degrees (10°). 

55. The apparatus as defined in Claim 46, wherein the local texture for each said point comprises 
photometric data at or proximal to the point. 
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